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Abstract 

The Integrated Technology Assessment Center (ITAC) has developed a flexible s V s ‘ e ms ana'ysis 
framework to identify long-term technology needs, quantify payoffs for technology mvM^ejte, and 
assess the progress of ASTP-sponsored technology programs in the hypersomcs area. For this IT/ ^ 
has assembled 9 an experienced team representing a broad sector of the aerospace community and 
developed a systematic assessment process complete with supporting tools. ^ 

svstems are selected based on relevance to the ASTP and integrated concept models (ICM) of these 
concepts are developed. Key technologies of interest are identified and projections are made _of thor 
characteristics with respect to their impacts on key aspects of the specific concepts of interest Both the 
models and technology projections are then fed into the ITAC’s probabilistic systems analysis framework 
in ModelCenter® This framework permits rapid sensitivity analysis, single point ass ® ssm ®'l t ' 

a fill probabilistic assessment of each concept with respect to both ^TS^^inat multivariate 
technologies. Probabilistic outputs are weighed against metrics of ^ To^ram is 

decision making process to provide inputs for technology prioritization within the ASTP. ITAC program is 
currently finishing the assessment of a two-stage-to-orbit (TSTO), rocket-based combined cycle (RBCC) 
concept and a TSTO turbine-based combined cycle (TBCC) concept developed by the team wi p 
from NASA A baseline all rocket TSTO concept is also being developed for comparison Boeing has 
Sn^brntoff performance model for their Flexible Aerospace System Soluton , for Tomorro» 

(FASST) concept and the ISAT program will provide inputs for a sin 9 le - s,a 9®;'°^ 

based concept in the near-term. Both of these latter concepts will be analyzed within the ITAC framework 

over the summer. This paper provides a status update of the ITAC program. 


Introduction 

The development of a new generation of reusable 
launch vehicles (RLV) capable of carrying 
significant payloads at low cost and with both rapid 
turn-around capabilities and safety margins well 
beyond those of current systems is an area of 
great interest both to NASA and the Department of 
Defense. Many of the hypersonic concepts now 
under study will require the development of 
airbreathing engine and other advanced 
technologies and new operational strategies. 
These RLV concepts are relatively far-term, 
however, and the impacts of advanced 
technologies and engineering methods on key 
system metrics are not well known. NASAs 
Advanced Space Transportation Program (ASTP) 
is responsible for making far-term investments in 
low technology readiness level (TRL) technologies 
(TRL < 6) for advanced hypersonic transportation 
systems. To help guide the investment process, 
ASTP supports the Integrated Technology 
Assessment Center (ITAC) at the National Space 
Science and Technology Center (NSSTC). ITAC 
has developed and demonstrated a flexible 
framework for systems analysis that identifies 
long-term technology needs, provides fast turn- 
around sensitivity and gap analyses, and 
quantifies technology investment payoffs in a 
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probabilistic fashion. The system can also be 
used to measure progress in ASTP-sponsored 
technology programs. ITAC has engaged a 
diverse team representing a broad cross section 
of the aerospace community, defined a 
sophisticated and comprehensive systems 
analysis process, developed the necessary tool 
set, and is currently implementing the process on 
a number of concepts selected by ASTP . 

The systems analysis task is accomplished 
through the use of the Framework for Advanced 
Systems Trade-Offs using Probabilistic Analysis of 
Concepts and Technologies (FASTPACT). 
FASTPACT is an evaluation framework based on 
an analyst-friendly analysis tool called 
ModelCenter®. Integrated Concept Models (ICM) 
characterize selected concepts and include 
modules describing performance, operations, 
cost, safety, business, and economics. For 
analysis, the ICM of interest is integrated into the 
FASTPACT framework along with estimates of the 
effects of technologies, Technology 
Characterization Models (TCM) of interest. Both 
the ICMs and the TCMs are generated from 
information obtained by the ITAC team from a 
variety of expert sources (both within and outside 
the team). The framework also incorporates 
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Monte Carlo, Optimization, Multi-Attribute Decision 
Making (MADM), and Genetic Algorithm (GA) 
analysis components. 

The outputs are probabilistic estimates of the 
impacts of technologies (singular or in suites). 
Output data is then processed in a sophisticated 
multi-variate decision making tool developed by 
ITAC, MADM, to provide quantified inputs on 
technology impacts (e.g. critical technologies for 
individual concepts, high impact technologies 
across concepts, and benefit-to-cost ratios). 
Customer-supplied metrics (and weightings) along 
with technology development cost estimates 
gleaned from available data and expert judgments 
are key inputs. These outputs also serve as a 
basis for comparisons in progress assessments 
as ASTP-sponsored development programs 
mature. Furthermore, the ITAC-generated ICMs 
can also be easily exercised in sensitivity analyses 
to rapidly identify high leverage areas and 
technology gaps. In addition, FASTPACT uses the 
GA component to rapidly assess suites of 
technologies and determine the suites that provide 
the biggest payoff as measured by the MADM 
module. 

The ITAC systems analysis process will be 
described in this paper, as will the concepts 
currently being processed, the schedule, and 
recent results. 

Current ITAC Concepts and Schedule 

The first three vehicle concepts being examined in 
the ITAC program are an all rocket, vertical take- 
off, horizontal landing (VTHL) two-stage-to-orbit 
(TSTO) system (ICM-1), a horizontal take-off^ 
horizontal landing (HTHL) TSTO system with a 1 
stage hydrogen (LH2) fueled turbine-based 
combination cycle propulsion system and a rocket- 
based upper stage (ICM-2), and a HTHL TSTO 
with a LOX/LH2 rocket-based combined cycle 1 
stage propulsion system and a rocket-based 
upper stage (ICM-3). 

The all rocket concept serves as a baseline for 
analysis and will incorporate features of interest 
from multiple past and present programs. 
Similarly, both HTHL TSTO concepts draw 
significant heritage from the NASA-sponsored 
Access-to-Space (ATS) study and the Air 
Breathing Launch Vehicle (ABLV) study . 

A top-level schedule for the ITAC program is 
shown in Figure 1 . 



Figure 1. ITAC Top Level Schedule 

This schedule is comprised of four segments: 

• ICM Development 

• Technology Characterization 

• Systems Analysis 

• Tools Development 

The reverse order of delivery in the numbered ICM 
in the April/June timeframe is due to changes in 
priority within ASTP. ICM-3 was delivered on 
schedule and is currently in the systems analysis 
process, and ICM-2 was nearing completion at the 
time of this writing. The results shown in this paper 
are initial results from the ICM-3 development 
efforts. 

In addition to the three concepts described above, 
the Boeing Aerospace Company (a key member 
of the ITAC team) has recently submitted the 
performance module for their Flexible Aerospace 
System Solution for Tomorrow (FASST) concept. 
This concept is likely to become ICM-4 on the 
schedule. Additionally, the data required to 
develop the performance module for a single- 
stage-to-orbit (SSTO), TBCC will shortly be 
delivered by NASA’s Integrated Systems Analysis 
Team (ISAT) and this will be the basis for ICM-5. 

ITAC Concept Models (ICMs) 

To rapidly assess technology tradeoffs for a 
particular concept in a probabilistic manner, the 
ITAC team has created fast-running parametric 
models of the vehicle concept. The model relates 
system output responses such as price per pound 
and turnaround time to input variables such as 
required vehicle parameters like payload, thrust- 
to-weight, propulsion system characteristics, cost 
factors, safety and reliability factors, etc. To 
generate the model, a concept point design is 
converted to a fast running parameterized format, 
the ICM, which is typically in the form of Microsoft 
Excel workbooks. The ICM can be thought of as 
similar to a system response surface model of the 
concept that is valid for a given region about the 
baseline reference (or operating) point. Each ICM 
is composed of a series of modules that represent 
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the overall system. These modules currently 
include a performance module (also contains the 
weights, sizing, and trajectory information), a cost 
module, an operations module, and a safety and 
reliability module. Each of these modules is 
discussed in more detail below. 

The performance module is actually composed of 
two parts - the weights and sizing model and the 
trajectory model. The weights and sizing model 
contains the weight, volume, and area equations 
for the vehicle systems and uses photographic 
scaling to re-size the vehicle as necessary to meet 
the mission mass ratio requirements. This scaling 
is iterative and is done automatically in Excel. The 
required mass ratios are determined from the 
trajectory analysis. Initially, an external trajectory 
code, Optimal Trajectories by Implicit Simulation 
(OTIS) version 3.1 is used to generate a closed 
trajectory to the desired orbit for the vehicle. This 
is an iterative process between the weights and 
sizing model and the trajectory code. Typically 
only two to three iterations are required to 
converge to a closed solution. Using this 
converged baseline or reference trajectory, a 
series of additional trajectories are calculated 
using thrust-to-weight and area-to-weight ratios as 
parameters to generate a trajectory response 
surface. The response surface can then be used 
to very quickly and easily examine the effects of 
changes in vehicle parameters due to new 
technologies. For a given vehicle weight, it can 
supply a required mass ratio to the weights model 
and thus the new weight and mass ratios are 
determined by iteration between the two models. 

The operations module is based on an Excel 
spreadsheet called The Architectural Assessment 
Tool - enhanced (AATe) 3 , developed by 
NASA/Kennedy Space Center, which is a tool for 
performing operations analysis and decision 
making support during the conceptual design 
phase of a reusable space transportation system. 
AATe can generate rapid estimates for both a 
reusable space transportation systems cost and 
cycle times from landing to launch. These 
estimates are generated based principally on the 
systems design and technology, which is input to 
the tool. AATe is used as a relative comparison 
tool to evaluate various vehicle configurations. 
Typical input values include vehicle dimensions 
and configuration, payload, vehicle reliability and 
design life, and the level of ops-related 
technologies like IVHM, etc. Outputs include 
vehicle turnaround time, fixed and variable 
operating cost, and facilities acquisition cost. 

The NASA-Air Force Cost Model (NAFCOM) 
developed by SAIC 4 is the basis for the cost 
module. NAFCOM consolidates numerous 
existing cost models and databases used 


throughout NASA. This fully automated software 
tool employs an easy-to-use spreadsheet 
environment to predict the cost of space hardware 
at the subsystem and component levels. The 
information within NAFCOM represents the best of 
the aerospace project data from the Resource 
Data Storage and Retrieval (REDSTAR) library, 
NASA's major repository of cost, technical, and 
programmatic information dating back to the 
1960s. Cost estimates created within NAFCOM 
are based on specific analogy and database 
averaging techniques. The user selects analogous 
data points from the database within NAFCOM to 
create specific analogy cost estimating 
relationships (CERs). After making these 
selections, the user further refines the CER 
database by choosing from more than 100 filters 
within the cost model that relate to the technical 
and programmatic characteristics of the data 
points. For the ITAC cost model NAFCOM was 
used to construct an Excel workbook which 
calculates costs and which contains the interface 
to other modules, primarily the weights which are 
used to calculate costs using the appropriate 
CERs. 

The safety and reliability module is based on a 
fault-tree analysis of vehicle failure modes. Failure 
modes analyzed include propulsion system 
failures (ejector failure, ramjet failure, scramjet 
failure, turbopump failures, engine cooling failures, 
etc.), TPS failures, separation system failures, and 
landing failures. 

The ITAC team has completed an RBCC-based 
TSTO ICM, and is completing a second TBCC- 
based TSTO ICM. Both vehicles are HTHL with 
an airbreathing propulsion system for the first 
stage and an all rocket second stage. Each of 
these vehicles is described in more detail below. 

ICM-3: RBCC-based TSTO Vehicle 

An ICM for the RBCC-based TSTO vehicle has 
been created. The assumed mission is to deliver a 
20,000 lb payload ina12ftx15ftx30ft payload 
bay to a 100 nautical mile orbit due east of Cape 
Kennedy. Both stages are un-crewed, but the 
payload can be either passengers or cargo. 

The mission can be summarized as follows: 

• Vehicle takes off from the launch site in ejector 
(rocket) mode 

• Vehicle transitions to ramjet mode and then 
scramjet mode 

• Ascends to staging Mach number (varies from 
Mach 6 to Mach 12) 

• Conducts a “pull up” maneuver to a dynamic 
pressure (q) of 500 psf 

• Powers off 1st stage propulsion and climbs until 
q = 200 psf 
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• Stages separate 

• 2nd stage continues by rocket power to orbit 

• 1st stage does an unpowered “turnaround” 
maneuver 

• 1st stage cruises under ramjet power (<M=6) 

• 1st stage propulsion powers off (M = 3 to 5) 

• 1st stage makes unpowered landing at the 
launch site 

• 2nd stage delivers payload 

• 2nd stage reenters atmosphere and returns to 
the launch site for an un-powered landing 

A weights and sizing model of this vehicle was 
constructed and used in conjunction with OTIS to 
create a closed trajectory. The first stage weight 
model was derived from the airbreathing/rocket 
SSTO vehicle that was designed in the NASA 
Access to Space (ATS) study 1 . It has a wedge- 
shaped fore body profile lifting-body configuration 
with all moving horizontal tails, twin vertical tails 
with rudders, and trailing edge body flaps. The 
LH2 tank is a cold graphite epoxy (Gr/Ep), integral, 
conformal, l-stiffened tank that contains 
pressurized fuel and carries the airframe loads. 
There is a non-integral aluminum lithium (Al/Li) 
LOX tank and a Gr/Ep shell structure fore and aft 
of the integral LH2 tank. There are two six-wheel 
main landing gear and a two-wheel nose gear. All 
moving horizontal controls and twin 
verticals/rudder are constructed of Titanium Matrix 
Composites. 

The thermal protection system (TPS) utilizes 
Fibrous Refractory Composite Insulation (FRCI- 
12) over Rohacel (cryogenic closed cell foam) with 
Toughened, Unipiece Fibrous Insulation (TUFI) 
coating used on the windward surface. Tailorable 
Advanced Blanket Insulation (TABI) over Rohacel 
with Protective Ceramic Coating (PCC-B) is used 
on the leeward surface. The tank insulation 
consists of Rohacel plus ceramic reusable surface 
insulation (RSI). The aerosurface TPS is 
Carbon/Silicon carbide (C/SiC) over portions 
exceeding 1960 deg R with carbon-carbon (C/C) 
leading and trailing edges. 

The ATS weights model was altered as follows to 
match the TSTO first stage requirements: 

• Removed external rocket system, crew cabin 
weights 

• Added structural provisions for 2nd stage 

• Added separation system weights 

• Altered propulsion system weight for RBCC 
engine 

Model volumes included fixed volume for 
subsystems (from ATS report), propellant tank 
volume, and volume for the second stage 
immersion. 


The second stage is also a lifting body shape and 
was based on a TSTO vehicle designed by 
NASA/Ames Research Center . Some 
modifications were made: 

• Added a body flap for hypersonic trim 
. Re-sized the aerosurfaces for maximum landing 
speed 

. Set aerosurface trailing edge sweep to 0 
degrees to eliminate adverse yaw due to roll 
control 

. Re-sized payload bay to 12’ x 15’ x 30’ 

The second stage also utilizes cold, integral, 
conformal Gr/Ep LH2 tanks and an Al-Li non- 
conformal LOX tank. The rocket engine is 
assumed to be similar to the Pratt and Whitney 
RL-X 6 and has an assumed thrust to weight of 70 
and specific impulse of Isp = 455 seconds. 
Separate OMS/RCS systems were assumed with 
associated tanks and lines. Electromechanical 
actuators were assumed for control surface 
actuation (no hydraulics). The Fuselage TPS 
consisted of Advanced Carbon-Carbon on the 
high temperature areas as well as TABI tiles. The 
tail uses Shuttle type Reusable Carbon-Carbon 
(RCC)/ceramic. The body flap and base also use 
shuttle type ceramic TPS. Dry weight margins of 
15% were used on both stages. A summary of the 
baseline vehicle weights is shown in Table 1 . 

Table 1. RBCC-based TSTO Vehicle Weights 


Item 

First Stage 

Second Stage 

Wing 

14,556 lb 

0 lb 

Tail 

2,814 lb 

1,874 lb 

Body 

88,140 lb 

15,575 lb 

TPS 

18,378 lb 

9,906 lb 

Undercarriage 

35,923 lb 

4,648 lb 

Propulsion 

35,601 lb 

9,929 lb 

Sub-Systems 

27,182 lb 

11,204 lb 

Dry Weight Margin (15%) 

39.281 It 

8.740 lb 

Dry Weight 

261,876 lb 

61,875 1b 

Payload 

348,251 lb 

20,000 lb 

Propellant (includes 

609.465 lb 

266.376 lb 

residuals and losses) 

Gross Weight 

1,219,592 lb 

348,251 lb 


The aerodynamic model of the first stage was 
based on data from another similar vehicle in a 
NASA/Langley Research Center study done by the 
Boeing Company 7 . Boeing generated the first 
stage propulsion data deck used in the trajectory 
analysis as well. The aerodynamic model of the 
second stage was created using the Aerodynamic 
Preliminary Analysis System (APAS) code. 

The baseline trajectory assumed that the vehicles 
staged at Mach 8. Trajectory analysis results are 
shown in Figures 2 and 3. 
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TSTO-RBCC Tr>]#et*fy 



Figure 2. Mach and Altitude vs. Time 


TSTO-RBCC Dynamic Praaiurt 



Figure 3. Altitude and Q vs. Time 

As described above, these results were used to 
create a trajectory response surface with 
parameters of thrust to weight (T/W) and surface 
area to weight (S/W). This response surface is 
shown graphically in Figure 4. 


ICM-3 Trajectory Reap on*# Surfaca (Madi 8 Separation) 



ICM-2 TBCC-based TSTO Vehicle 

The second vehicle being studied is very similar to 
the RBCC-based TSTO except for the I s stage 
propulsion system. A Turbine Based Combination 
Cycle (TBCC) engine that consists of an over- 
under arrangement with the hydrogen-fueled 
turbojet engines located above the 
Ramjet/Scramjets is used. The turbojets operate 
from takeoff until the Ramjet mode is started and 
the sequence of events is then very similar to the 
RBCC vehicle. However, after separation the 
turbojets are eventually restarted to allow a 
powered descent and landing. 

Except for the differences due to the addition of 
the turbojets, the layout and weights of this vehicle 
are similar to the RBCC. A detailed analysis of this 
vehicle is still underway so a baseline weights and 
trajectory statement is not yet available. 

ITAC Systems Analysis 

The systems analysis framework leverages 
commercial-off-the-shelf software to create an 
easily configured set of components that are 
integrated to form a rapidly configurable set of 
analyses. The Excel Workbooks used to create 
the ICM modules are integrated into the 
ModelCenter® framework using wrappers to 
expose the interface variables. A typical analysis 
is depicted in Figure 5. 



Figure 5. N-squared Diagram for ICM3 in 
FASTPACT 


Figure 4. Typical Trajectory Response Surface 

The response surface equations were then 
integrated into the closure module of the ICM. 
Based on initial outputs, a second set of trajectory 
runs were performed and a second response 
surface equation developed so that analyses for 
Mach 10 staging could also be performed. 


In this figure we see all of the components of 
ICM3 and the FASTPCT framework. First across 
the diagonal from upper left towards lower right 
are the major ICM-3 components. Each model 
component represents the Excel spreadsheet that 
performs the calculations for that module. Then 
along the left side are three Monte Carlo 
components specific to economic, concept 
analysis, and technology analysis disciplines. 
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Next, the technology impact analysis is performed 
using components in the lower left portion of the 
figure — these include the TCM, GA Study, MADM 
and ITAC GA Monte Carlo modules. Finally on the 
right side are configurable data monitors that 
display a condensed list of parameters and values 
in real-time as the model is run. 

The first sets of studies conducted were sensitivity 
analyses to examine and verify the proper 
functioning of all the model components. 
Typically, each component within the ICM is 
studied individually to determine the value range 
of input and output variables so that these results 
can be verified against the modelers’ expected 
values. In most instances, the process begins 
with a screening analysis using a Placket-Burman 
or Taguchi type design of experiments to identify 
the variable main effects within the component. 
Determining the variable main effects allows the 
analyst to reduce the number of variables within 
the design of experiments so that a more 
complete set of runs can be conducted. 

Once the variables of interest are selected, a full 
or fractional factorial analysis of these variables is 
conducted to determine the main and second 
order effects on output variables of interest. For 
ASTP, the key metrics are Cost, Safety, and 
Operations related. Typical results of sensitivity 
analyses on key metrics of interest to ASTP are 
shown in Figures 6 and 7 . 


Parameter 

Main Effect 

Percent Impact 

Fixed Recurring Cost 
Factor 

3222.71 

22% 

2 nd Stage l SP 

-2905.23 

20% 

Variable Recurring 
Cost Factor 

2809.59 

19% 

TFU 1 f1 Stage Engine 

393.893 

2% 

2" d Stage Landing 
Reliability 

-307.783 

2% 


Nol*: other factors nol presented bring total percent Impact to 100% 


Figure 6. Subset of Factors That Affect 
Recurring Cost Per Pound 


Parameter 

Main Effect 

Percent Impact 

2 nd Stage l SP 

348360 

27% 

Stage Hypersonic 

*SP 

-149228 

11% 

2 nd Stage TPS Weight 

90257.8 

7% 

V Stage Takeoff- 
Machl Igp 

-66707.5 

5% 

I s ' Stage Machl- 
Mach3 Igp 

-65406.1 

5% 


Not*: othw factors not prestntad bring total poreont Impact to 100% 


Figure 7. Subset of Factors That Affect Gross 
Takeoff Weight 

Once all the components are found to perform as 
expected, they are integrated into the analysis 
framework with the analytic components. These 
components provide the means to perform 
extensive parametric and probabilistic analysis. 
The first sets of analyses conducted are the single 
technology impact studies, where each technology 
is overlaid onto the concept vehicle and run via a 
Monte Carlo process to determine the distribution 
of key metrics output by the model. 

Following the sensitivity analyses. Technology 
Characterization Models (TCM) are used to 
capture and describe mathematically the effect a 
specified technology has on a specified concept. 
The TCMs are developed by ITAC through 
researching the technology, discussions with 
technologists, and evaluation by boards of 
experts. The TCM consists of a collection of input 
distributions for ICM input variables that reflect the 
impact a certain technology could have on these 
variables and the uncertainty assessed for these 
outcomes. 

For example, a new engine technology could 
improve the trajectory-averaged l sp and thrust to 
weight ratio (T/W) of the baseline engine from its 
expected values of 450 seconds and 20. If the 
technology is expected to produce 550 seconds of 
l sp but could produce as much as 700 seconds or 
as few as 450 seconds, the input could be 
modeled as a triangular distribution with a low 
value of 450, a peak value of 550, and a high 
value of 700. 

Similarly, if T/W is expected to be 25 with the new 
technology but could vary symmetrically around 
that point, the effect on T/W could be modeled as 
a normal distribution with a mean of 25 and a 
standard deviation of 5. The technology is 
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overlaid on the concept by replacing the existing 
input variables with the TCM distributions and 
conducting a significant number of Monte Carlo 
simulation runs to collect the performance output 
metrics. From this analysis we are able to 
determine the impact a single technology could 
have on the performance of the concept, as 
measured by the ASTP metrics. 

A key portion of the technology analyses is the 
multivariate decision making portion of the 
process. This portion of the analysis allows the 
performance metrics to be combined into a single 
value based on the ASTP goals for technology-to- 
technology comparisons. Using standard 
Analytical Hierarchy Process (AHP) techniques, 
IT AC can elicit customer, ASTP, value judgments 
for each of the metrics of interest. Regression 
analysis recreates these value judgments in the 
form of an equation (Saaty Function) that provides 
the customer’s judgment across the range of 
possible values. For example, we determined 
ASTP’s perceived value of a variety of price per 
pound of payload outcomes and generated the 
Saaty Function where the Saaty Value is the value 
ASTP placed on a particular price per pound 
outcome. The 3 rd Generation RLV goal of $100 
per pound produces a value of 8 where the scores 
range from a minimum of 1 to a maximum of 9. 
The curve fit of these judged values provides a 
ready means of mapping performance variable 
outcomes into the decision maker’s value space. 
In this example, the curve fit produced the 
function: 

Score = -1 .3026 ln(x) + 1 4 

where x is the price per pound metric determined 
by the model outputs. Applying similar logic, we 
are able to determine the Saaty Functions for 
each metric and an appropriate weighting among 
the metrics to produce a weighted average score 
for any set of metric outputs from the model. 

In a similar vein, ITAC conducts multiple- 
technology analyses to determine the impact of a 
collection or portfolio of technologies on the 
concept vehicle. When the number of 
technologies is sufficiently small, exhaustive 
enumeration of the combinations is used to 
generate the test cases; otherwise a heuristic 
optimization routine (genetic algorithm) is used to 
intelligently select portfolios for analysis. The 
genetic algorithm (GA) is coded to turn on or off 
the technology candidates to form a population of 
portfolios. 

These portfolios are then evaluated using the 
same logic as used in the single technology 
analysis methodology with one notable exception, 
the convolution of the TCM distributions into a 


single set of input distributions. When multiple 
technologies are considered for analysis, the 
likelihood of several technologies impacting the 
same input variable is increased. In those 
instances, a method for convolving or combining 
the input distributions is required. 

Once the inputs for a portfolio are convolved, the 
analysis proceeds as described above to produce 
output distributions of performance metrics and 
value scores. These scores form the fitness 
evaluation for the population of portfolios and 
influence the selection of portfolios allowed to 
procreate for the next generation in the GA. 

Clearly the weighting of the outcomes is a 
significant driver for technology selection. 
Therefore, a sensitivity analysis is conducted to 
see how the technology selections vary as the 
weights among the metrics are changed. A 
sample output from this analysis for a technology 
portfolio analysis is shown in Figure 8. 



Figure 8. Portfolio Analysis Affected by 
Weightings 

This type of analysis provides users the ability to 
determine the sensitivity of the technology portfolio 
selection to variations in the metric weights. Here 
it is apparent that the weighting of Safety - 50%, 
Cost - 25% and Operations - 25% produces a 
portfolio 64 (P64) selection and that reducing the 
Safety weight to 35% causes a change in 
preference to a different portfolio. 

The final analytic technique available to ITAC is 
benefit-to-cost analysis. This analysis should not 
be confused with the cost modeling of the ICM or 
with the price per pound metric produced by the 
previous analysis. This analysis attempts to 
capture the benefits of procuring one or more 
technologies and balance that with the potential 
risks with that procurement. In these analyses the 
benefit portion of the result is generated as 
described above, the weighted score value for the 
technology or portfolio. The calculation of the cost 
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portion of the metric involves collecting additional 
data to develop a weighted score value for cost as 
was done for benefit. ITAC has used cost metrics 
involving R&D Costs, R&D Degree of Difficulty, 
Technology Readiness Levels, and Schedule to 
develop an AHP based cost methodology. The 
cost metrics are evaluated and Saaty Functions 
are developed for each to provide a mapping from 
the performance domain into the value domain. In 
this instance low Saaty Scores are preferable to 
higher scores. Once the functions are defined and 
the data collected to populate the functions, 
weighted cost scores are combined to produce an 
overall cost value. The benefit score value and 
the cost score value for a particular technology or 
portfolio are divided to provide a benefit-to-cost 
ratio. This ratio captures the gains as well as risks 
or potential problems with a technology selection. 
This balances the two to provide an overall 
assessment of which technologies or portfolios 
provide the most gain for the least risk. 

Other analyses are possible using the analysis 
framework developed. Technology trade-offs, 
concept evaluations, concept optimizations, and 
what-if analyses of many types are possible. 
Continued growth and development of these tools 
and techniques can provide higher levels of 
modeling fidelity and more tailored products to 
support the ASTP program now and well into the 
future. The value these techniques and tools 
provide is the rapid means of determining effects 
and the ability to map those effects into terms 
relevant to the ASTP goals and decision making 
processes. 

Summary and Directions 

The Integrated Technology Assessment Center 
has developed a flexible framework to provide 
comprehensive systems analysis to assist in the 
guidance of ASTP technology development and 
demonstration programs. A number of RLV 
concepts based on airbreathing propulsion 
systems are currently in process along with an all- 


rocket vehicle for comparison purposes. Results 
of the first set of probabilistic analyses are nearing 
completion and it is anticipated that a large suite 
of concepts will be assessed over the next year. 
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